

<!DOCTYPE html>
<html class="writer-html5" lang="en" >
<head>
  <meta charset="utf-8" />
  <meta name="generator" content="Docutils 0.19: https://docutils.sourceforge.io/" />

  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
  
  <title>迁移到 BlueStore &mdash; Ceph Documentation</title>
  

  
  <link rel="stylesheet" href="../../../_static/ceph.css" type="text/css" />
  <link rel="stylesheet" href="../../../_static/pygments.css" type="text/css" />
  <link rel="stylesheet" href="../../../_static/pygments.css" type="text/css" />
  <link rel="stylesheet" href="../../../_static/ceph.css" type="text/css" />
  <link rel="stylesheet" href="../../../_static/graphviz.css" type="text/css" />
  <link rel="stylesheet" href="../../../_static/css/custom.css" type="text/css" />

  
  

  
  

  

  
  <!--[if lt IE 9]>
    <script src="../../../_static/js/html5shiv.min.js"></script>
  <![endif]-->
  
    
      <script type="text/javascript" id="documentation_options" data-url_root="../../../" src="../../../_static/documentation_options.js"></script>
        <script src="../../../_static/jquery.js"></script>
        <script src="../../../_static/_sphinx_javascript_frameworks_compat.js"></script>
        <script data-url_root="../../../" id="documentation_options" src="../../../_static/documentation_options.js"></script>
        <script src="../../../_static/doctools.js"></script>
        <script src="../../../_static/sphinx_highlight.js"></script>
    
    <script type="text/javascript" src="../../../_static/js/theme.js"></script>

    
    <link rel="index" title="Index" href="../../../genindex/" />
    <link rel="search" title="Search" href="../../../search/" />
    <link rel="next" title="控制命令" href="../control/" />
    <link rel="prev" title="设备管理" href="../devices/" /> 
</head>

<body class="wy-body-for-nav">

   
  <header class="top-bar">
    <div role="navigation" aria-label="Page navigation">
  <ul class="wy-breadcrumbs">
      <li><a href="../../../" class="icon icon-home" aria-label="Home"></a></li>
          <li class="breadcrumb-item"><a href="../../">Ceph 存储集群</a></li>
          <li class="breadcrumb-item"><a href="../">集群运维</a></li>
      <li class="breadcrumb-item active">迁移到 BlueStore</li>
      <li class="wy-breadcrumbs-aside">
            <a href="../../../_sources/rados/operations/bluestore-migration.rst.txt" rel="nofollow"> View page source</a>
      </li>
  </ul>
  <hr/>
</div>
  </header>
  <div class="wy-grid-for-nav">
    
    <nav data-toggle="wy-nav-shift" class="wy-nav-side">
      <div class="wy-side-scroll">
        <div class="wy-side-nav-search"  style="background: #eee" >
          

          
            <a href="../../../" class="icon icon-home"> Ceph
          

          
          </a>

          

          
<div role="search">
  <form id="rtd-search-form" class="wy-form" action="../../../search/" method="get">
    <input type="text" name="q" placeholder="Search docs" aria-label="Search docs" />
    <input type="hidden" name="check_keywords" value="yes" />
    <input type="hidden" name="area" value="default" />
  </form>
</div>

          
        </div>

        
        <div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation">
          
            
            
              
            
            
              <ul class="current">
<li class="toctree-l1"><a class="reference internal" href="../../../start/">Ceph 简介</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../install/">安装 Ceph</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../cephadm/">Cephadm</a></li>
<li class="toctree-l1 current"><a class="reference internal" href="../../">Ceph 存储集群</a><ul class="current">
<li class="toctree-l2"><a class="reference internal" href="../../configuration/">配置</a></li>
<li class="toctree-l2 current"><a class="reference internal" href="../">运维</a><ul class="current">
<li class="toctree-l3"><a class="reference internal" href="../operating/">操纵集群</a></li>
<li class="toctree-l3"><a class="reference internal" href="../health-checks/">健康检查</a></li>
<li class="toctree-l3"><a class="reference internal" href="../monitoring/">监控集群</a></li>
<li class="toctree-l3"><a class="reference internal" href="../monitoring-osd-pg/">监控 OSD 和归置组</a></li>
<li class="toctree-l3"><a class="reference internal" href="../user-management/">用户管理</a></li>
<li class="toctree-l3"><a class="reference internal" href="../pgcalc/">PG Calc</a></li>
<li class="toctree-l3"><a class="reference internal" href="../data-placement/">数据归置概览</a></li>
<li class="toctree-l3"><a class="reference internal" href="../pools/">存储池</a></li>
<li class="toctree-l3"><a class="reference internal" href="../erasure-code/">纠删码</a></li>
<li class="toctree-l3"><a class="reference internal" href="../cache-tiering/">分级缓存</a></li>
<li class="toctree-l3"><a class="reference internal" href="../placement-groups/">归置组</a></li>
<li class="toctree-l3"><a class="reference internal" href="../upmap/">使用 pg-upmap</a></li>
<li class="toctree-l3"><a class="reference internal" href="../read-balancer/">Operating the Read (Primary) Balancer</a></li>
<li class="toctree-l3"><a class="reference internal" href="../balancer/">均衡器模块</a></li>
<li class="toctree-l3"><a class="reference internal" href="../crush-map/">CRUSH 图</a></li>
<li class="toctree-l3"><a class="reference internal" href="../crush-map-edits/">手动编辑一个 CRUSH 图</a></li>
<li class="toctree-l3"><a class="reference internal" href="../stretch-mode/">Stretch Clusters</a></li>
<li class="toctree-l3"><a class="reference internal" href="../change-mon-elections/">Configuring Monitor Election Strategies</a></li>
<li class="toctree-l3"><a class="reference internal" href="../add-or-rm-osds/">增加/删除 OSD</a></li>
<li class="toctree-l3"><a class="reference internal" href="../add-or-rm-mons/">增加/删除监视器</a></li>
<li class="toctree-l3"><a class="reference internal" href="../devices/">设备管理</a></li>
<li class="toctree-l3 current"><a class="current reference internal" href="#">迁移到 BlueStore</a><ul>
<li class="toctree-l4"><a class="reference internal" href="#bluestore-osd">用 BlueStore 部署新 OSD</a></li>
<li class="toctree-l4"><a class="reference internal" href="#osd">转换已有 OSD</a></li>
</ul>
</li>
<li class="toctree-l3"><a class="reference internal" href="../control/">命令参考</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../troubleshooting/community/">Ceph 社区</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../troubleshooting/troubleshooting-mon/">监视器故障排除</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../troubleshooting/troubleshooting-osd/">OSD 故障排除</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../troubleshooting/troubleshooting-pg/">归置组排障</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../troubleshooting/log-and-debug/">日志记录和调试</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../troubleshooting/cpu-profiling/">CPU 剖析</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../troubleshooting/memory-profiling/">内存剖析</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="../../man/">    手册页</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../troubleshooting/">故障排除</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../api/">APIs</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../../../cephfs/">Ceph 文件系统</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../rbd/">Ceph 块设备</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../radosgw/">Ceph 对象网关</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../mgr/">Ceph 管理器守护进程</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../mgr/dashboard/">Ceph 仪表盘</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../monitoring/">监控概览</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../api/">API 文档</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../architecture/">体系结构</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../dev/developer_guide/">开发者指南</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../dev/internals/">Ceph 内幕</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../governance/">项目管理</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../foundation/">Ceph 基金会</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../ceph-volume/">ceph-volume</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../releases/general/">Ceph 版本（总目录）</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../releases/">Ceph 版本（索引）</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../security/">Security</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../hardware-monitoring/">硬件监控</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../glossary/">Ceph 术语</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../jaegertracing/">Tracing</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../translation_cn/">中文版翻译资源</a></li>
</ul>

            
          
        </div>
        
      </div>
    </nav>

    <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap">

      
      <nav class="wy-nav-top" aria-label="top navigation">
        
          <i data-toggle="wy-nav-top" class="fa fa-bars"></i>
          <a href="../../../">Ceph</a>
        
      </nav>


      <div class="wy-nav-content">
        
        <div class="rst-content">
        
          <div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
           <div itemprop="articleBody">
            
<div id="dev-warning" class="admonition note">
  <p class="first admonition-title">Notice</p>
  <p class="last">This document is for a development version of Ceph.</p>
</div>
  <div id="docubetter" align="right" style="padding: 5px; font-weight: bold;">
    <a href="https://pad.ceph.com/p/Report_Documentation_Bugs">Report a Documentation Bug</a>
  </div>

  
  <section id="bluestore">
<span id="rados-operations-bluestore-migration"></span><h1>迁移到 BlueStore<a class="headerlink" href="#bluestore" title="Permalink to this heading"></a></h1>
<div class="admonition warning">
<p class="admonition-title">Warning</p>
<p>Filestore 在 Reef 版已经废弃了，且不再支持。请迁移到 BlueStore 。</p>
</div>
<p>Each OSD must be formatted as either Filestore or BlueStore. However, a Ceph
cluster can operate with a mixture of both Filestore OSDs and BlueStore OSDs.
Because BlueStore is superior to Filestore in performance and robustness, and
because Filestore is not supported by Ceph releases beginning with Reef, users
deploying Filestore OSDs should transition to BlueStore. There are several
strategies for making the transition to BlueStore.</p>
<p>BlueStore is so different from Filestore that an individual OSD cannot be
converted in place. Instead, the conversion process must use either (1) the
cluster’s normal replication and healing support, or (2) tools and strategies
that copy OSD content from an old (Filestore) device to a new (BlueStore) one.</p>
<section id="bluestore-osd">
<h2>用 BlueStore 部署新 OSD<a class="headerlink" href="#bluestore-osd" title="Permalink to this heading"></a></h2>
<p>Use BlueStore when deploying new OSDs (for example, when the cluster is
expanded). Because this is the default behavior, no specific change is
needed.</p>
<p>Similarly, use BlueStore for any OSDs that have been reprovisioned after
a failed drive was replaced.</p>
</section>
<section id="osd">
<h2>转换已有 OSD<a class="headerlink" href="#osd" title="Permalink to this heading"></a></h2>
<section id="out">
<h3>“标记为 <code class="docutils literal notranslate"><span class="pre">out</span></code> ”的替代<a class="headerlink" href="#out" title="Permalink to this heading"></a></h3>
<p>The simplest approach is to verify that the cluster is healthy and
then follow these steps for each Filestore OSD in succession: mark the OSD
<code class="docutils literal notranslate"><span class="pre">out</span></code>, wait for the data to replicate across the cluster, reprovision the OSD,
mark the OSD back <code class="docutils literal notranslate"><span class="pre">in</span></code>, and wait for recovery to complete before proceeding
to the next OSD. This approach is easy to automate, but it entails unnecessary
data migration that carries costs in time and SSD wear.</p>
<ol class="arabic">
<li><p>Identify a Filestore OSD to replace:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">ID</span><span class="o">=&lt;</span><span class="n">osd</span><span class="o">-</span><span class="nb">id</span><span class="o">-</span><span class="n">number</span><span class="o">&gt;</span>
<span class="n">DEVICE</span><span class="o">=&lt;</span><span class="n">disk</span><span class="o">-</span><span class="n">device</span><span class="o">&gt;</span>
</pre></div>
</div>
<ol class="arabic">
<li><p>Determine whether a given OSD is Filestore or BlueStore:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><style type="text/css">
span.prompt1:before {
  content: "$ ";
}
</style><span class="prompt1">ceph<span class="w"> </span>osd<span class="w"> </span>metadata<span class="w"> </span><span class="nv">$ID</span><span class="w"> </span><span class="p">|</span><span class="w"> </span>grep<span class="w"> </span>osd_objectstore</span>
</pre></div></div></li>
<li><p>Get a current count of Filestore and BlueStore OSDs:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">ceph<span class="w"> </span>osd<span class="w"> </span>count-metadata<span class="w"> </span>osd_objectstore</span>
</pre></div></div></li>
</ol>
</li>
<li><p>Mark a Filestore OSD <code class="docutils literal notranslate"><span class="pre">out</span></code>:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">ceph<span class="w"> </span>osd<span class="w"> </span>out<span class="w"> </span><span class="nv">$ID</span></span>
</pre></div></div></li>
<li><p>Wait for the data to migrate off this OSD:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1"><span class="k">while</span><span class="w"> </span>!<span class="w"> </span>ceph<span class="w"> </span>osd<span class="w"> </span>safe-to-destroy<span class="w"> </span><span class="nv">$ID</span><span class="w"> </span><span class="p">;</span><span class="w"> </span><span class="k">do</span><span class="w"> </span>sleep<span class="w"> </span><span class="m">60</span><span class="w"> </span><span class="p">;</span><span class="w"> </span><span class="k">done</span></span>
</pre></div></div></li>
<li><p>Stop the OSD:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">systemctl<span class="w"> </span><span class="nb">kill</span><span class="w"> </span>ceph-osd@<span class="nv">$ID</span></span>
</pre></div></div></li>
<li id="osd-id-retrieval"><p>Note which device the OSD is using:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">mount<span class="w"> </span><span class="p">|</span><span class="w"> </span>grep<span class="w"> </span>/var/lib/ceph/osd/ceph-<span class="nv">$ID</span></span>
</pre></div></div></li>
<li><p>Unmount the OSD:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">umount<span class="w"> </span>/var/lib/ceph/osd/ceph-<span class="nv">$ID</span></span>
</pre></div></div></li>
<li><p>Destroy the OSD’s data. Be <em>EXTREMELY CAREFUL</em>! These commands will destroy
the contents of the device; you must be certain that the data on the device is
not needed (in other words, that the cluster is healthy) before proceeding:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">ceph-volume<span class="w"> </span>lvm<span class="w"> </span>zap<span class="w"> </span><span class="nv">$DEVICE</span></span>
</pre></div></div></li>
<li><p>Tell the cluster that the OSD has been destroyed (and that a new OSD can be
reprovisioned with the same OSD ID):</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">ceph<span class="w"> </span>osd<span class="w"> </span>destroy<span class="w"> </span><span class="nv">$ID</span><span class="w"> </span>--yes-i-really-mean-it</span>
</pre></div></div></li>
<li><p>Provision a BlueStore OSD in place by using the same OSD ID. This requires
you to identify which device to wipe, and to make certain that you target
the correct and intended device, using the information that was retrieved in
the <a class="reference internal" href="#osd-id-retrieval"><span class="std std-ref">“Note which device the OSD is using”</span></a> step.  BE
CAREFUL!  Note that you may need to modify these commands when dealing with
hybrid OSDs:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">ceph-volume<span class="w"> </span>lvm<span class="w"> </span>create<span class="w"> </span>--bluestore<span class="w"> </span>--data<span class="w"> </span><span class="nv">$DEVICE</span><span class="w"> </span>--osd-id<span class="w"> </span><span class="nv">$ID</span></span>
</pre></div></div></li>
<li><p>Repeat.</p></li>
</ol>
<p>You may opt to (1) have the balancing of the replacement BlueStore OSD take
place concurrently with the draining of the next Filestore OSD, or instead
(2) follow the same procedure for multiple OSDs in parallel. In either case,
however, you must ensure that the cluster is fully clean (in other words, that
all data has all replicas) before destroying any OSDs. If you opt to reprovision
multiple OSDs in parallel, be <strong>very</strong> careful to destroy OSDs only within a
single CRUSH failure domain (for example, <code class="docutils literal notranslate"><span class="pre">host</span></code> or <code class="docutils literal notranslate"><span class="pre">rack</span></code>). Failure to
satisfy this requirement will reduce the redundancy and availability of your
data and increase the risk of data loss (or even guarantee data loss).</p>
<p>Advantages:</p>
<ul class="simple">
<li><p>Simple.</p></li>
<li><p>Can be done on a device-by-device basis.</p></li>
<li><p>No spare devices or hosts are required.</p></li>
</ul>
<p>Disadvantages:</p>
<ul class="simple">
<li><p>Data is copied over the network twice: once to another OSD in the cluster (to
maintain the specified number of replicas), and again back to the
reprovisioned BlueStore OSD.</p></li>
</ul>
</section>
<section id="id1">
<h3>“整机”替换<a class="headerlink" href="#id1" title="Permalink to this heading"></a></h3>
<p>If you have a spare host in the cluster, or sufficient free space to evacuate
an entire host for use as a spare, then the conversion can be done on a
host-by-host basis so that each stored copy of the data is migrated only once.</p>
<p>To use this approach, you need an empty host that has no OSDs provisioned.
There are two ways to do this: either by using a new, empty host that is not
yet part of the cluster, or by offloading data from an existing host that is
already part of the cluster.</p>
<section id="id2">
<h4>利用新的、空主机<a class="headerlink" href="#id2" title="Permalink to this heading"></a></h4>
<p>Ideally the host will have roughly the same capacity as each of the other hosts
you will be converting.  Add the host to the CRUSH hierarchy, but do not attach
it to the root:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1"><span class="nv">NEWHOST</span><span class="o">=</span>&lt;empty-host-name&gt;</span>
<span class="prompt1">ceph<span class="w"> </span>osd<span class="w"> </span>crush<span class="w"> </span>add-bucket<span class="w"> </span><span class="nv">$NEWHOST</span><span class="w"> </span>host</span>
</pre></div></div><p>Make sure that Ceph packages are installed on the new host.</p>
</section>
<section id="id3">
<h4>利用已有主机<a class="headerlink" href="#id3" title="Permalink to this heading"></a></h4>
<p>If you would like to use an existing host that is already part of the cluster,
and if there is sufficient free space on that host so that all of its data can
be migrated off to other cluster hosts, you can do the following (instead of
using a new, empty host):</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1"><span class="nv">OLDHOST</span><span class="o">=</span>&lt;existing-cluster-host-to-offload&gt;</span>
<span class="prompt1">ceph<span class="w"> </span>osd<span class="w"> </span>crush<span class="w"> </span>unlink<span class="w"> </span><span class="nv">$OLDHOST</span><span class="w"> </span>default</span>
</pre></div></div><p>where “default” is the immediate ancestor in the CRUSH map. (For
smaller clusters with unmodified configurations this will normally
be “default”, but it might instead be a rack name.) You should now
see the host at the top of the OSD tree output with no parent:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">bin/ceph<span class="w"> </span>osd<span class="w"> </span>tree</span>
</pre></div></div><div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">ID</span> <span class="n">CLASS</span> <span class="n">WEIGHT</span>  <span class="n">TYPE</span> <span class="n">NAME</span>     <span class="n">STATUS</span> <span class="n">REWEIGHT</span> <span class="n">PRI</span><span class="o">-</span><span class="n">AFF</span>
<span class="o">-</span><span class="mi">5</span>             <span class="mi">0</span> <span class="n">host</span> <span class="n">oldhost</span>
<span class="mi">10</span>   <span class="n">ssd</span> <span class="mf">1.00000</span>     <span class="n">osd</span><span class="mf">.10</span>        <span class="n">up</span>  <span class="mf">1.00000</span> <span class="mf">1.00000</span>
<span class="mi">11</span>   <span class="n">ssd</span> <span class="mf">1.00000</span>     <span class="n">osd</span><span class="mf">.11</span>        <span class="n">up</span>  <span class="mf">1.00000</span> <span class="mf">1.00000</span>
<span class="mi">12</span>   <span class="n">ssd</span> <span class="mf">1.00000</span>     <span class="n">osd</span><span class="mf">.12</span>        <span class="n">up</span>  <span class="mf">1.00000</span> <span class="mf">1.00000</span>
<span class="o">-</span><span class="mi">1</span>       <span class="mf">3.00000</span> <span class="n">root</span> <span class="n">default</span>
<span class="o">-</span><span class="mi">2</span>       <span class="mf">3.00000</span>     <span class="n">host</span> <span class="n">foo</span>
 <span class="mi">0</span>   <span class="n">ssd</span> <span class="mf">1.00000</span>         <span class="n">osd</span><span class="mf">.0</span>     <span class="n">up</span>  <span class="mf">1.00000</span> <span class="mf">1.00000</span>
 <span class="mi">1</span>   <span class="n">ssd</span> <span class="mf">1.00000</span>         <span class="n">osd</span><span class="mf">.1</span>     <span class="n">up</span>  <span class="mf">1.00000</span> <span class="mf">1.00000</span>
 <span class="mi">2</span>   <span class="n">ssd</span> <span class="mf">1.00000</span>         <span class="n">osd</span><span class="mf">.2</span>     <span class="n">up</span>  <span class="mf">1.00000</span> <span class="mf">1.00000</span>
<span class="o">...</span>
</pre></div>
</div>
<p>If everything looks good, jump directly to the <a class="reference internal" href="#bluestore-data-migration-step"><span class="std std-ref">“Wait for the data
migration to complete”</span></a> step below and proceed
from there to clean up the old OSDs.</p>
</section>
<section id="id4">
<h4>迁移过程<a class="headerlink" href="#id4" title="Permalink to this heading"></a></h4>
<p>If you’re using a new host, start at <a class="reference internal" href="#bluestore-migration-process-first-step"><span class="std std-ref">the first step</span></a>. If you’re using an existing host,
jump to <a class="reference internal" href="#bluestore-data-migration-step"><span class="std std-ref">this step</span></a>.</p>
<ol class="arabic" id="bluestore-migration-process-first-step">
<li><p>Provision new BlueStore OSDs for all devices:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">ceph-volume<span class="w"> </span>lvm<span class="w"> </span>create<span class="w"> </span>--bluestore<span class="w"> </span>--data<span class="w"> </span>/dev/<span class="nv">$DEVICE</span></span>
</pre></div></div></li>
<li><p>核对一下这个 OSD 已加入集群：</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">ceph<span class="w"> </span>osd<span class="w"> </span>tree</span>
</pre></div></div><p>你应该能看到这个新主机 <code class="docutils literal notranslate"><span class="pre">$NEWHOST</span></code> ，它下面还挂着所有的
OSD ，但是这台主机<em>不应该</em>嵌入分级结构的其它任何节点（像 <code class="docutils literal notranslate"><span class="pre">root</span> <span class="pre">default</span></code> ）。例如，假设 <code class="docutils literal notranslate"><span class="pre">newhost</span></code> 就是这台空主机，你可能看到类似的：</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ bin/ceph osd tree
ID CLASS WEIGHT  TYPE NAME     STATUS REWEIGHT PRI-AFF
-5             0 host newhost
10   ssd 1.00000     osd.10        up  1.00000 1.00000
11   ssd 1.00000     osd.11        up  1.00000 1.00000
12   ssd 1.00000     osd.12        up  1.00000 1.00000
-1       3.00000 root default
-2       3.00000     host oldhost1
 0   ssd 1.00000         osd.0     up  1.00000 1.00000
 1   ssd 1.00000         osd.1     up  1.00000 1.00000
 2   ssd 1.00000         osd.2     up  1.00000 1.00000
...
</pre></div>
</div>
</li>
<li><p>Identify the first target host to convert:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1"><span class="nv">OLDHOST</span><span class="o">=</span>&lt;existing-cluster-host-to-convert&gt;</span>
</pre></div></div></li>
<li><p>Swap the new host into the old host’s position in the cluster:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">ceph<span class="w"> </span>osd<span class="w"> </span>crush<span class="w"> </span>swap-bucket<span class="w"> </span><span class="nv">$NEWHOST</span><span class="w"> </span><span class="nv">$OLDHOST</span></span>
</pre></div></div><p>At this point all data on <code class="docutils literal notranslate"><span class="pre">$OLDHOST</span></code> will begin migrating to the OSDs on
<code class="docutils literal notranslate"><span class="pre">$NEWHOST</span></code>.  If there is a difference between the total capacity of the
old hosts and the total capacity of the new hosts, you may also see some
data migrate to or from other nodes in the cluster. Provided that the hosts
are similarly sized, however, this will be a relatively small amount of
data.</p>
</li>
<li id="bluestore-data-migration-step"><p>Wait for the data migration to complete:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1"><span class="k">while</span><span class="w"> </span>!<span class="w"> </span>ceph<span class="w"> </span>osd<span class="w"> </span>safe-to-destroy<span class="w"> </span><span class="k">$(</span>ceph<span class="w"> </span>osd<span class="w"> </span>ls-tree<span class="w"> </span><span class="nv">$OLDHOST</span><span class="k">)</span><span class="p">;</span><span class="w"> </span><span class="k">do</span><span class="w"> </span>sleep<span class="w"> </span><span class="m">60</span><span class="w"> </span><span class="p">;</span><span class="w"> </span><span class="k">done</span></span>
</pre></div></div></li>
<li><p>Stop all old OSDs on the now-empty <code class="docutils literal notranslate"><span class="pre">$OLDHOST</span></code>:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">ssh<span class="w"> </span><span class="nv">$OLDHOST</span></span>
<span class="prompt1">systemctl<span class="w"> </span><span class="nb">kill</span><span class="w"> </span>ceph-osd.target</span>
<span class="prompt1">umount<span class="w"> </span>/var/lib/ceph/osd/ceph-*</span>
</pre></div></div></li>
<li><p>Destroy and purge the old OSDs:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1"><span class="k">for</span><span class="w"> </span>osd<span class="w"> </span><span class="k">in</span><span class="w"> </span><span class="sb">`</span>ceph<span class="w"> </span>osd<span class="w"> </span>ls-tree<span class="w"> </span><span class="nv">$OLDHOST</span><span class="sb">`</span><span class="p">;</span><span class="w"> </span><span class="k">do</span></span>
<span class="prompt1"><span class="w">   </span>ceph<span class="w"> </span>osd<span class="w"> </span>purge<span class="w"> </span><span class="nv">$osd</span><span class="w"> </span>--yes-i-really-mean-it</span>
<span class="prompt1"><span class="k">done</span></span>
</pre></div></div></li>
<li><p>Wipe the old OSDs. This requires you to identify which devices are to be
wiped manually. BE CAREFUL! For each device:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">ceph-volume<span class="w"> </span>lvm<span class="w"> </span>zap<span class="w"> </span><span class="nv">$DEVICE</span></span>
</pre></div></div></li>
<li><p>Use the now-empty host as the new host, and repeat:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1"><span class="nv">NEWHOST</span><span class="o">=</span><span class="nv">$OLDHOST</span></span>
</pre></div></div></li>
</ol>
<p>Advantages:</p>
<ul class="simple">
<li><p>Data is copied over the network only once.</p></li>
<li><p>An entire host’s OSDs are converted at once.</p></li>
<li><p>Can be parallelized, to make possible the conversion of multiple hosts at the same time.</p></li>
<li><p>No host involved in this process needs to have a spare device.</p></li>
</ul>
<p>Disadvantages:</p>
<ul class="simple">
<li><p>A spare host is required.</p></li>
<li><p>An entire host’s worth of OSDs will be migrating data at a time. This
is likely to impact overall cluster performance.</p></li>
<li><p>All migrated data still makes one full hop over the network.</p></li>
</ul>
</section>
</section>
<section id="per-osd-device-copy">
<h3>Per-OSD device copy<a class="headerlink" href="#per-osd-device-copy" title="Permalink to this heading"></a></h3>
<p>A single logical OSD can be converted by using the <code class="docutils literal notranslate"><span class="pre">copy</span></code> function
included in <code class="docutils literal notranslate"><span class="pre">ceph-objectstore-tool</span></code>. This requires that the host have one or more free
devices to provision a new, empty BlueStore OSD. For
example, if each host in your cluster has twelve OSDs, then you need a
thirteenth unused OSD so that each OSD can be converted before the
previous OSD is reclaimed to convert the next OSD.</p>
<p>Caveats:</p>
<ul class="simple">
<li><p>This approach requires that we prepare an empty BlueStore OSD but that we do not allocate
a new OSD ID to it. The <code class="docutils literal notranslate"><span class="pre">ceph-volume</span></code> tool does not support such an operation. <strong>IMPORTANT:</strong>
because the setup of <em>dmcrypt</em> is closely tied to the identity of the OSD, this approach does not
work with encrypted OSDs.</p></li>
<li><p>The device must be manually partitioned.</p></li>
<li><p>An unsupported user-contributed script that demonstrates this process may be found here:
<a class="reference external" href="https://github.com/ceph/ceph/blob/master/src/script/contrib/ceph-migrate-bluestore.bash">https://github.com/ceph/ceph/blob/master/src/script/contrib/ceph-migrate-bluestore.bash</a></p></li>
</ul>
<p>优点：</p>
<ul class="simple">
<li><p>转换期间，只要 OSD 或集群上设置了 <cite>noout</cite> 或者 <cite>norecover</cite>/<cite>norebalance</cite>  标记，
就只会有少量或没有数据通过网络迁移。</p></li>
</ul>
<p>缺点：</p>
<ul class="simple">
<li><p>工具链尚未完全实现、支持、或有文档；</p></li>
<li><p>Each host must have an appropriate spare or empty device for staging.</p></li>
<li><p>The OSD is offline during the conversion, which means new writes to PGs
with the OSD in their acting set may not be ideally redundant until the
subject OSD comes up and recovers. This increases the risk of data
loss due to an overlapping failure.  However, if another OSD fails before
conversion and start-up are complete, the original Filestore OSD can be
started to provide access to its original data.</p></li>
</ul>
</section>
</section>
</section>



<div id="support-the-ceph-foundation" class="admonition note">
  <p class="first admonition-title">Brought to you by the Ceph Foundation</p>
  <p class="last">The Ceph Documentation is a community resource funded and hosted by the non-profit <a href="https://ceph.io/en/foundation/">Ceph Foundation</a>. If you would like to support this and our other efforts, please consider <a href="https://ceph.io/en/foundation/join/">joining now</a>.</p>
</div>


           </div>
           
          </div>
          <footer><div class="rst-footer-buttons" role="navigation" aria-label="Footer">
        <a href="../devices/" class="btn btn-neutral float-left" title="设备管理" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left" aria-hidden="true"></span> Previous</a>
        <a href="../control/" class="btn btn-neutral float-right" title="控制命令" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right" aria-hidden="true"></span></a>
    </div>

  <hr/>

  <div role="contentinfo">
    <p>&#169; Copyright 2016, Ceph authors and contributors. Licensed under Creative Commons Attribution Share Alike 3.0 (CC-BY-SA-3.0).</p>
  </div>

   

</footer>
        </div>
      </div>

    </section>

  </div>
  

  <script type="text/javascript">
      jQuery(function () {
          SphinxRtdTheme.Navigation.enable(true);
      });
  </script>

  
  
    
   

</body>
</html>